{
  "cells": [
    {
      "cell_type": "raw",
      "id": "afaf8039",
      "metadata": {
        "vscode": {
          "languageId": "raw"
        }
      },
      "source": [
        "---\n",
        "sidebar_label: Groq\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e49f1e0d",
      "metadata": {},
      "source": [
        "# ChatGroq\n",
        "\n",
        "[Groq](https://groq.com/) is a company that offers fast AI inference, powered by LPU™ AI inference technology which delivers fast, affordable, and energy efficient AI.\n",
        "\n",
        "This will help you getting started with ChatGroq [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatGroq features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html).\n",
        "\n",
        "## Overview\n",
        "### Integration details\n",
        "\n",
        "| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/docs/integrations/chat/groq) | Package downloads | Package latest |\n",
        "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
        "| [ChatGroq](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html) | [`@langchain/groq`](https://www.npmjs.com/package/@langchain/groq) | ❌ | ❌ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/groq?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/groq?style=flat-square&label=%20&) |\n",
        "\n",
        "### Model features\n",
        "\n",
        "See the links in the table headers below for guides on how to use specific features.\n",
        "\n",
        "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
        "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: |\n",
        "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | \n",
        "\n",
        "## Setup\n",
        "\n",
        "To access ChatGroq models you'll need to create a Groq account, get an API key, and install the `@langchain/groq` integration package.\n",
        "\n",
        "### Credentials\n",
        "\n",
        "In order to use the Groq API you'll need an API key. You can sign up for a Groq account and create an API key [here](https://wow.groq.com/).\n",
        "Then, you can set the API key as an environment variable in your terminal:\n",
        "\n",
        "```bash\n",
        "export GROQ_API_KEY=\"your-api-key\"\n",
        "```\n",
        "\n",
        "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
        "\n",
        "```bash\n",
        "# export LANGSMITH_TRACING=\"true\"\n",
        "# export LANGSMITH_API_KEY=\"your-api-key\"\n",
        "```\n",
        "\n",
        "### Installation\n",
        "\n",
        "The LangChain ChatGroq integration lives in the `@langchain/groq` package:\n",
        "\n",
        "```{=mdx}\n",
        "\n",
        "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
        "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
        "\n",
        "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
        "\n",
        "<Npm2Yarn>\n",
        "  @langchain/groq @langchain/core\n",
        "</Npm2Yarn>\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a38cde65-254d-4219-a441-068766c0d4b5",
      "metadata": {},
      "source": [
        "## Instantiation\n",
        "\n",
        "Now we can instantiate our model object and generate chat completions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
      "metadata": {},
      "outputs": [],
      "source": [
        "import { ChatGroq } from \"@langchain/groq\" \n",
        "\n",
        "const llm = new ChatGroq({\n",
        "    model: \"llama-3.3-70b-versatile\",\n",
        "    temperature: 0,\n",
        "    maxTokens: undefined,\n",
        "    maxRetries: 2,\n",
        "    // other params...\n",
        "})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2b4f3e15",
      "metadata": {},
      "source": [
        "## Invocation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "62e0dbc3",
      "metadata": {
        "tags": []
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "AIMessage {\n",
            "  \"content\": \"I enjoy programming. (The French translation is: \\\"J'aime programmer.\\\")\\n\\nNote: I chose to translate \\\"I love programming\\\" as \\\"J'aime programmer\\\" instead of \\\"Je suis amoureux de programmer\\\" because the latter has a romantic connotation that is not present in the original English sentence.\",\n",
            "  \"additional_kwargs\": {},\n",
            "  \"response_metadata\": {\n",
            "    \"tokenUsage\": {\n",
            "      \"completionTokens\": 73,\n",
            "      \"promptTokens\": 31,\n",
            "      \"totalTokens\": 104\n",
            "    },\n",
            "    \"finish_reason\": \"stop\"\n",
            "  },\n",
            "  \"tool_calls\": [],\n",
            "  \"invalid_tool_calls\": []\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "const aiMsg = await llm.invoke([\n",
        "    {\n",
        "      role: \"system\",\n",
        "      content: \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
        "    },\n",
        "    { role: \"user\", content: \"I love programming.\" },\n",
        "])\n",
        "aiMsg"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "I enjoy programming. (The French translation is: \"J'aime programmer.\")\n",
            "\n",
            "Note: I chose to translate \"I love programming\" as \"J'aime programmer\" instead of \"Je suis amoureux de programmer\" because the latter has a romantic connotation that is not present in the original English sentence.\n"
          ]
        }
      ],
      "source": [
        "console.log(aiMsg.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ce0414fe",
      "metadata": {},
      "source": [
        "## Json invocation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "3f0a7a2a",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  aiInvokeMsgContent: '{\\n\"result\": 6\\n}',\n",
            "  aiBindMsg: '{\\n\"result\": 6\\n}'\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "const messages = [\n",
        "  {\n",
        "    role: \"system\",\n",
        "    content: \"You are a math tutor that handles math exercises and makes output in json in format { result: number }.\",\n",
        "  },\n",
        "  { role: \"user\",  content: \"2 + 2 * 2\" },\n",
        "];\n",
        "\n",
        "const aiInvokeMsg = await llm.invoke(messages, { response_format: { type: \"json_object\" } });\n",
        "\n",
        "// if you want not to pass response_format in every invoke, you can bind it to the instance\n",
        "const llmWithResponseFormat = llm.bind({ response_format: { type: \"json_object\" } });\n",
        "const aiBindMsg = await llmWithResponseFormat.invoke(messages);\n",
        "\n",
        "// they are the same\n",
        "console.log({ aiInvokeMsgContent: aiInvokeMsg.content, aiBindMsg: aiBindMsg.content });"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
      "metadata": {},
      "source": [
        "## Chaining\n",
        "\n",
        "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "AIMessage {\n",
            "  \"content\": \"That's great! I can help you translate English phrases related to programming into German.\\n\\n\\\"I love programming\\\" can be translated to German as \\\"Ich liebe Programmieren\\\".\\n\\nHere are some more programming-related phrases translated into German:\\n\\n* \\\"Programming language\\\" = \\\"Programmiersprache\\\"\\n* \\\"Code\\\" = \\\"Code\\\"\\n* \\\"Variable\\\" = \\\"Variable\\\"\\n* \\\"Function\\\" = \\\"Funktion\\\"\\n* \\\"Array\\\" = \\\"Array\\\"\\n* \\\"Object-oriented programming\\\" = \\\"Objektorientierte Programmierung\\\"\\n* \\\"Algorithm\\\" = \\\"Algorithmus\\\"\\n* \\\"Data structure\\\" = \\\"Datenstruktur\\\"\\n* \\\"Debugging\\\" = \\\"Debuggen\\\"\\n* \\\"Compile\\\" = \\\"Kompilieren\\\"\\n* \\\"Link\\\" = \\\"Verknüpfen\\\"\\n* \\\"Run\\\" = \\\"Ausführen\\\"\\n* \\\"Test\\\" = \\\"Testen\\\"\\n* \\\"Deploy\\\" = \\\"Bereitstellen\\\"\\n* \\\"Version control\\\" = \\\"Versionskontrolle\\\"\\n* \\\"Open source\\\" = \\\"Open Source\\\"\\n* \\\"Software development\\\" = \\\"Softwareentwicklung\\\"\\n* \\\"Agile methodology\\\" = \\\"Agile Methodik\\\"\\n* \\\"DevOps\\\" = \\\"DevOps\\\"\\n* \\\"Cloud computing\\\" = \\\"Cloud Computing\\\"\\n\\nI hope this helps! Let me know if you have any other questions or if you need further translations.\",\n",
            "  \"additional_kwargs\": {},\n",
            "  \"response_metadata\": {\n",
            "    \"tokenUsage\": {\n",
            "      \"completionTokens\": 327,\n",
            "      \"promptTokens\": 25,\n",
            "      \"totalTokens\": 352\n",
            "    },\n",
            "    \"finish_reason\": \"stop\"\n",
            "  },\n",
            "  \"tool_calls\": [],\n",
            "  \"invalid_tool_calls\": []\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "import { ChatPromptTemplate } from \"@langchain/core/prompts\"\n",
        "\n",
        "const prompt = ChatPromptTemplate.fromMessages(\n",
        "    [\n",
        "        [\n",
        "            \"system\",\n",
        "            \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
        "        ],\n",
        "        [\"human\", \"{input}\"],\n",
        "    ]\n",
        ")\n",
        "\n",
        "const chain = prompt.pipe(llm);\n",
        "await chain.invoke(\n",
        "    {\n",
        "        input_language: \"English\",\n",
        "        output_language: \"German\",\n",
        "        input: \"I love programming.\",\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
      "metadata": {},
      "source": [
        "## API reference\n",
        "\n",
        "For detailed documentation of all ChatGroq features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "TypeScript",
      "language": "typescript",
      "name": "tslab"
    },
    "language_info": {
      "codemirror_mode": {
        "mode": "typescript",
        "name": "javascript",
        "typescript": true
      },
      "file_extension": ".ts",
      "mimetype": "text/typescript",
      "name": "typescript",
      "version": "3.7.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
